Multimodal Engagement Analysis from Facial Videos in the Classroom

Student engagement is a key component of learning and teaching, resulting in plethora automated methods to measure it. Whereas most the literature explores student analysis using computer-based often lab, we focus on classroom instruction authentic environments. We collected audiovisual recordings secondary school classes over one half month period, acquired continuous labeling per (N=15) repeated sessions, explored computer vision classify from facial videos. learned deep embeddings for attentional affective features by training Attention-Net head pose estimation Affect-Net expression recognition previously-collected large-scale datasets. used these representations train classifiers our data, individual multiple channel settings, considering temporal dependencies. The best performing achieved student-independent AUCs .620 .720 grades 8 12, respectively, with attention-based outperforming features. Score-level fusion either improved or was par modality. also investigated effect personalization found that only 60 seconds person-specific selected margin uncertainty base classifier, yielded an average AUC improvement .084.